1. Field of the Invention
This invention relates to synchronous data mirrors and more particularly relates to efficiently swapping from a primary storage volume to a synchronous data mirror in system that includes an asynchronous data mirror.
2. Description of the Related Art
Data storage system users may require a high degree of reliability and quick recovery from failures. Some data storage system users may require protection from data loss due to natural disasters. For example, data storage system users such as financial institutions, retailers, and the like may demand a high degree of reliability for financial transactions, inventory control, etc. An enterprises storage system (“ESS”), such as the TotalStorage® Enterprise Storage Server® from IBM®, may be used by financial institutions and others to store data where reliability is important. Such systems may include highly scalable data storage devices organized into a data storage volume, such as a redundant array of inexpensive/independent disks (“RAID”).
A data storage system may include a storage area network (“SAN”) connecting data storage volumes. Typically a storage controller is used to access data storage volumes over the SAN. A host typically sends write requests to a storage controller which stores the data on a local, primary storage volume. In a cascaded data system arrangement, the storage controller also sends the data stored on the primary storage volume to a storage controller of a secondary data storage volume. Data storage volumes may be also located a long distance from the primary storage volume and associated storage controller to allow recovery after a natural disaster that affects the primary storage volume. Data storage volumes may also be relatively close to the primary storage volume, which reduces data access rates.
To minimize recovery time after a failure, a secondary storage volume may be a mirror of the primary data volume. Data may be written to the mirror data volume using a synchronous write operation. Typically, a synchronous write operation requires that data be written to the primary storage volume and the mirror data volume before success is returned to a host requesting the write operation. A synchronous write operation requires more overhead due to the time required to successfully write data to a primary storage volume and to the mirror storage volume.
Typically, distance between the primary storage volume and the mirror data volume affects performance so that data storage volumes located a long distance from the primary storage volume and associated storage controller may degrade performance enough to make synchronous write operations impractical. Asynchronous write operations may be used to write data to a remote data volume. An asynchronous write operation typically allows success to be returned for a write operation after the data is successfully written to the storage volume where the write operation is directed. The data is then written to the remote storage volume after completion of writing data to the storage volume storage where the write operation is directed.
In a cascaded storage system, data may be written to a primary storage volume and synchronously to a mirror data volume. The data is then copied asynchronously from the mirror data volume to a remote data volume. Data intended for storage in a remote storage volume typically is stored in a non-volatile memory, or write cache as well as in the general cache of a storage controller so that the data is not lost if an error occurs before the data can be asynchronously written to the remote storage volume.
One or more bit maps are typically used to track data to be written asynchronously to a remote storage volume. An array of bits, often referred to as an active track array, changed track array, or change recording (“CR”) bitmap, typically is used to keep a real-time record by track address on the storage volume initiating the asynchronous transfer of changes made since the last synchronization, or last time the storage volume initiating the asynchronous transfer and the remote storage volume were consistent. A second bitmap, sometimes called a recovery track array, a copy track array, or out of sync (“OOS”) bitmap, is used to designate which tracks have been written to the primary storage volume as part of an asynchronous write operation and still need to be copied to the remote storage volume.
Typically, as updates are successfully written to the remote storage volume, corresponding bits of the OOS bitmap are cleared. In one embodiment, contents of the CR bitmap are periodically merged with the OOS bitmap. In another embodiment, contents of the CR bitmap are periodically written over the OOS bitmap, typically when the OOS bitmap indicates there are no more updates to be written to the remote storage volume.
A common data storage system architecture 100 is depicted in FIG. 1A. The data storage system 100 is embodied by a host 102 that sends data to a first storage controller 104 of a first storage volume 106. The first storage volume 106 is initially designated as a primary storage volume and receives data from the first storage controller 104 over a local connection 108. A second storage volume 110 is a mirror data volume of the primary storage volume and receives data over a synchronous connection 112 through an associated second storage controller 111. In one embodiment, the synchronous connection 112 is a Metro Mirror connection. Typically the first storage controller 104 and first storage volume 106 are located at a local site, but may also be separated by some distance.
Updates received by the second storage volume 110 are also written to a third storage volume 114, typically located remotely from the first and second storage volumes 106, 110. Updates are asynchronously sent to the third storage volume 114. The second and third storage volumes 110, 114 communicate over a connection 116 that may include fiber optic cables, copper cables, a wireless connection, etc. In one embodiment, the asynchronous connection 116 is a Global Mirror connection. A fourth storage volume 118, in one embodiment, is located with the third storage volume 114. A third storage controller 120 facilitates copying data to the third storage volume 114 and from the third storage volume 114 to the fourth storage volume 118. Typically, the third storage controller 120 copies updates to the fourth storage volume 118 using a flash copy process 122 and updates are assembled in a chronological order on the fourth storage volume 118.
Typically, the fourth storage volume 118, if present in the storage system 100, is used to restore data to the first or second storage volume 106, 110 if data is lost, corrupted, or inaccessible. In one embodiment, the contents of the fourth storage volume 118 are used to bring the third volume to a synchronous state with the third storage volume 114 and the third storage volume 114 is used to reinitialize the first and/or second storage volumes 106, 110. Replacing the data on the first or second storage volumes 106,110, rather than incrementally synchronizing the storage volumes 106, 110, 118 is undesirable due to the time required for the copy process and the time that the storage volumes 106, 110, 118 are inaccessible to the host 102.
The second storage volume 110 may have an associated CR bitmap 124 to track updates that happened after a time when the first and second storage volumes 106, 110 are consistent. The second storage volume 110 may also have an associated OOS bitmap to track when updates intended to be asynchronously copied to the third storage volume 114 are successfully copied. The OOS bitmap typically tracks updates by receiving the contents of the CR bitmap or updates received by the second storage volume 110.
FIG. 1B depicts a situation with a failure on the first storage volume 106. The host 102 typically swaps the primary storage volumes so that the second storage volume 110 is the primary storage volume until the first storage volume 106 is repaired. The host 102 typically establishes a connection 128 with the second storage volume 110. When the first storage volume 106 is repaired, as depicted in FIG. 1C, the second storage controller 111 of the second storage volume 110 typically suspends updates to the third storage volume 114 and establishes a connection 130 with the first storage volume 106 to resynchronize. In one embodiment, establishing a connection 130 to send data from the second storage volume 110 to the first storage volume 106 is a failover connection.
Once the first storage volume 106 is consistent with the second storage volume 110, as depicted in FIG. 1D, the host 102 swaps the primary storage volume to be the first storage volume 106 and the first storage volume 106 reestablishes the synchronous connection 112 with the second storage volume 110. In one embodiment, reestablishing sending data from the first storage volume 106 to the second storage volume 110 is a failback connection. The second storage volume 110 starts to again send updates asynchronously to the third storage volume 114, which begins to update the fourth storage volume 118. Failover and failback connections require pre-existing relationships, such as the storage relationship between the first storage volume 106 and the second storage volume 110.
During the time when the second storage volume 110 is synchronizing with the first storage volume 106 and has suspended updates to the third storage volume 114 (see FIG. 1C), the third and fourth storage volumes 114, 118 begin to age. If the first storage volume 106 was not operational for a long period of time, the third and fourth storage volumes 114, 118 could require a long time to be brought to a consistent state with the first and second storage volumes 106, 110. The process of re-synchronizing the first, third, and fourth storage volumes 106, 114, 118 could take hours or even days. Any disaster that requires the contents of the third storage volume 114 to be reloaded onto the first or second storage volume 106, 110 before the third storage volume 114 is brought to a nearly consistent state is problematic because the third storage volume 114 is not close to being consistent.
From the foregoing discussion, it should be apparent that a need exists for an apparatus, system, and method that efficiently swap synchronous mirror volumes. Beneficially, such an apparatus, system, and method would re-synchronize a first storage volume 106 while maintaining the second, third, and fourth storage volumes 110, 114, 118 in a nearly synchronous state and would decrease the time required to resynchronize the system 100. The apparatus, system, and method would force a failover between storage volumes where no pre-existing storage relationship existed previously.